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ABSTRACT 

Proceedings of the first meeting of the State 
Advisory Committee for the Pennsylvania Bureau of Quality Education 
Assessment are provided. The proceedings arc comprised of the 
following: Superintendent's Message; Charge to the Committee; The 
Plan for Assessing Educational Quality in Pennsylvania; vihat will 
Quality Education Assessment Measure?; Progress Report on the 
National Assessment Project; Summary of Comments and Suggestions; and 
Conclusion* Concerns of the committee vere related to the sampling 
procedure for the April 1968 testing, the drawback of the plan in the 
reliance on paper and pencil self-report instruments, the items 
language in terms of difficulty level and of semantics, the working 
of certain itens which seemed to imply a value judgment of the 
behavior in question, and the need to establish a list of 
^^guidelines'* for teachers concerning the nature of the assessment and 
its purposes. (DB) 
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The dol i bornt inns ot the SCnlc Advisory Conmiltee on the osscssmont 
oC oducntlonnl quall'cy represent the first steps toward systemnt icil ly 
determining the quality o; the educ^itlonnl prn^^rcTrr^s in tho public slIiooIs 
of the Commonwealth. 

It is my feeling that the Quality Education Assossnient Program is 
one of the most ouCstandinj^ programs in the Department ot Public 
Instruction. Difficult as the task nay be the results, I am certain, 
will be most rewarding. As we move toward an educational system of 
excellence tlirough the improvement of instruction, all citizens of tho 
Commonweal til will benefit. 

A program on paper is one kind of plan. The implementation of the 
program is that part of Quality Education Assessment which we are 
emphasizing in these Proceedings. If we are to have objective evidence 
about the adequacy and efficiency of educational programs, with emphasis 
on the individual school and its capacity for meeting the specific needs 
of the specific children it is to serve, we must take the necessary 
first step toward reaching this ultimate goal. Together I am sure we 
^ will reach this goal. 

Wo appreciate the comments and suggestions the Committee has given 
us, and we are grateful for its continuing advice and support In our 
joint effort* 

David H. Kurtzman 

erJc , 



CHARGE TO COMMITTEE 



Follow educators and friends of education, since I assumed the 
responsibilities of the Office of Commissioner for Basic Education Pve 
been doing my best to discourage unnecessary meetings. Mo try, whenever 
possible, to rely on memoranda and telephones, rather than waste precious 
hours of busy people by scheduling another meeting, except for a very 
good reason. 

I tell you this to underline the significance with which we, in the 
Department, view this meeting of the State Advisory Committee for the 
Bureau of Quality Education Assessment which Is holding its initial 
meeting here today* This is a Blue Ribbon Committee, not only because 
it represents a cross-section of top leadership in Pennsylvania Education, 
but because its deliberations and suggestions relate to a concern among 
educators that portends nationrl significance In the matter of evaluating 
the quality of educational product in this Commonwealth. Pennsylvania can 
truly become a pioneer in the relatively untraveled frontier of assessment 
of quality education. 

You may remember that when the Department Reorganization was 
announced, the duties of the newly created post of Commissioner for Basic 
Education were clearly enumerated in the DPI Bulletin. 
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Of the iHciny bro,id responsibilities assigned to this office, tlie 
Lronscendont function, it Kccmod, was that "It shall bo his goal to move 
Basic Education to the highest quality possible In Pennsylvania^** This 
same objecLive happens to be one of the major concerns of the present 
Administration of Pennsylvania* 

One section of the far reaching School District Reorganization Act 
of 1963, Act 299, sets rorth the General Assembly's desire for Educational 
Performance Standards In Pennsvl vanl.i, as follows: 

"The Slate Board of Education • ♦ • • shall develop 
or cause to be developed an evaluation procedure designed to 
measure objectively the adequacy and efficiency of the 
educational programs offered by the public schools of the 
Commonweal th» The evaluation procedure to be developed shall 
include tests measuring the achievements and performance of 
students pursuing all of the various subjects and courses 
comprising the curricula • • • •" 

To carry out this mandate, the State Board appointed from its members 
a Committee on Quality Education, headed by Mrs» Albert M. Greenfield, who 
was also the chairman of the Board's Council of Basic Education. Upon this 
Coimnitteo's recommendation, the Board entered into a contract with the 
Educational Testing Service of Princeton, New Jersey, to assist in the 
development of a plan to implement this provision of Act 299. 

A series of studies during 1965 for this purpose resulted in a report 
by the Educational Testing Service to the Board. I will not take the time 
to suimnarize this report. You will find it in the Highlight Booklet you 
x-eceivcd at registration. 



In the lecontly completed reorganization of the Department of Public 
Instruct: ion« a Bureau of Quality Educ.itlon Assossmont has been created to 
spearhead this program, cooperatively with the Bureaus for Research, 
Curriculum, Guidance, Instructional Services and other related activities, 
and in concert with administrators throughout the Commonwealth. 

Even without a legislative mandate, the case for ^educational 
assessment seems to present a pursuasive argument these days. When we 
are spending 32 billion dollars of federal, state, and local tax money 
for education this year, more than the gross national product In 1935, 
It seems reasonable to expect that educators and taxpayers alike should 
want some demonstrable proof that the product we turn out Is Improving 
In quality. 

The functions spelled out for our Bureau of Quality Education 
Assessment are: 

In cooperation with school districts and 
Intermediate units, assess quality education 
In the public schools. 

Coordinate with other Bureaus In developing 
through research, experimentation and 
evaluation Instruments for measuring quality 
education. 

Develop procedures for applying measures of 
quality education In cooperation with local 
school districts. 

Analyze and report to the State Board of 
Education on quality education development. 

I have read these charges to the Bureau purposely to emphasize the 
repetitive use of the words "cooperation" and "coordination." 
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These are the keys to the success oC this new program of Quality Assessment. 
As .1 practitioner with 32 years in public education, I, like you, would view 
with alarm any state test arbitrarily Imposed on the schools. 

It Is not the purpose of the Department to assume an inspectorial or 
regulatory function. Rather, we envision this service as a cooperative 
venture to be shared and developed by all organizational levels of educational 
concern. 

He will work on the basis of programs and facilities now available in 
our schools. We will measure skills and achievements of children and teachers. 
We also hope to develop instruments to determine attitudes of administrators, 
teachers and pupils. 

Accomplishment of this goal will require the best thinking and the 
combined judgment of all educational leaders in our schools, as well as 
of the Department of Public Instruction professionals. 

To this end I beseech your understanding and your help. 

This, then, is our purpose and our hope. The charge to this Committee 
Is to ask for your help by giving us your thoughtful understanding, your 
professional judgment and your sympathetic concern^ 



B. Anton Hess 



THE PL/\N FOR ASSESSING 
EDUCATIONAL QUALITY IN PENNSYLVANIA 



Pennsylvania's Plan for educational assessment is of necessity and 
with full intrent based upon the planning and thinking of professionals 
who have been previously concerned with problems of a like nature* Isaac 
Newton's comment about standing on the shoulders of others applied whole- 
sale to our efforts* Our Plan utilizes, where possible, the work under- 
taken for the State Board of Education by the Educational Testing Service 
(l). It has been modified from the ETS Plan in order to fit more closely 
with the working realities of the Department of Public Instruction and of 
the public schools in the Commonwealth, It attempts to take into account 
the experience of Ralph Tyler and his associates with their plans for the 
national assessment of education, particularly those facets which caused 
so much concern among school administrators (2), The experience gained by 
our colleagues who have been engaged In the New York Quality Measurement 
Project has also been considered. The Plan reflects the position represented 
by Alexander Mood concerning the necessity of using, in many Instances, 
approximate, empirical and only logically valid devices. These are Items 
or exercises which look and sound as If they would measure the kind of 
achievement we want, although they have not been subjected to rigorous 
trial (3)# We agree with Dr. Mood that wo cannot wait until a final polished 
Instrument Is available for every area which we wish to measure* 



Wo are willing to begin with imperfect instruments because we think that by 
their very use, refinements in the instruments themselves will be possible. 
We are attempting to measure a broad range of achievements to avoid directing 
educational outcome toward some limited, test defined objective. 

The Pennsylvania Plan as presently conceived consists of three major 
t«isks* The first of these is the process of selecting or developing measurement 
devices for student output* Output Is defined as those behaviors which arc 
within a srudent's range of accomplishment at a given point In his school 
career. Can Johnio accurately multiply 429 x 325? Does he know where to find 
out about the requirements of a certain college or career? Does he have enotigh 
confidence In himself to try a difficult but possible task? Measurement devices 
must be selected or devised for each of the ten goals which were accepted by the 
Quality Education Committee of the State Board of Education In 1965. These 
goals stated In brief form arc: 

1. Self understanding and self acceptance. 

2. Understanding and appreciation of social, cultural, and ethnic 
groups different from his own« 

3. Mastery of the use of words and numbers. 

4. Positive attitudes toward school and learning. 

5. The habits and values associated with responsible citizenship. 

6. Good health habits and knowledge of the conditions necessary 
for maintaining physical and emotional well-being. 

7. Experience with creative processes in a variety of fields. 

8. A full grasp of the opportunities for preparing for a 
productive life. 

9. Understanding and appreciation of the natural sciences, tho 
social sciences, the humanities, and the arts. 

10. Preparation for effective participation In a changing world. 
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Dcponding upon the state of tho <irt, moasurcmont will consist ot tho 
dcgroG to vhlch a student Is able to porfonn bohjvlors which ciro consistent 
with, noccssciry to, or sufficient for, tho accomplishment of these goals* 
Continued discussion of the goals themselves among lay leaders and professional 
educators Is also required. 

The second part of tho Plan Involves classifying student output according 
to modifying conditions. A rovlov of the literature suggests that factors 
such as financial resources, community effort, average socioeconomic level, 
median teacher salary schedule, tcachcr/pupll ratio, and factors of student 
potential such as the ubiquitous IQ, Individual :;ocloccoi4omlc level, as well 
as mother*s educational level, all arc correlates ol output in slgtilf leant 
ways. In other words, the higher the median teacher salary schedule, thr 
higher tho student*s socioeconomic level, the higher his mother*s educational 
level, the more likely he is to score well on an output measure* 

We Intend to report our data In tho form of multiple classification 
oxpectnncy tables* What we moan by this can be Illustrated as follows: 

l^esources You know well the problems of finding adequate financial 
resources to operate the kind of schools you want for your communities* If 
we divide the school districts In tho state Into three categories, on tho basis 
of financial resources, we would expect to have at least three dlffciont 
distributions of student output for each of tliesc conditions* It*s asking a 
bit much to expect tho same output from a school with a $10,000* tax base 
behind each pupil as wo might expect from n school with $100,000* 
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Effort Anothor factor which may well influence output and truiy be 
lndopendont« or £it least partially independent* of resources is the affiount of 
money the consnunity is willing to put into education* or* an indax oi cofmrninlty 
effort. One wealthy district may put a much smaller proportion of its total 
resources into education than another* equally wealthy* Chi the other hand* 
a very poor district may be putting a very largo proportion of its resources 
into education* and* therefore* be able to compare favorably financially with 
another district more affluent than Itself* If such a variable as this is 
considered* wc have addlMonal groupings possible* For example* we might have 
a category of districts with high resources and low effort* or medium effort 
and I'-sv resources* Neither of these two variables is dlroctly controllable 
by the rulnol district Itself* although* x>£ course* the school may influence 
to some degree what the community Is willing to do* 

School Characteristics The third -7<:toi* v^lch may Influence output 
has to do wlU: the characteristics of the school itself* Those Include such 
things as the proportion of the school *s finances It puts Into Instruction 
as compared to overhead Items such as maintenance* administration* transportation* 
etc* Schools cannot completely control these factors* but can probably have 
more Influence In this area than In the amount of resources a community is 
willing to devote to education* For example* student/ instructor ratios would 
show up In this category as well as staffing patterns* educational level of 
staff members* etc* 

If we divide the rcnge of school characteristics into three parts* we 
should now have many more possible sub-populations for which different sots of 
expected output* or norms* might occur* 
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Student Potent t at The fourth ktnd of variable which needs to be taken 
into .account id that which the student hi^etf brings as an Individual* his own 
potential* At this point we propose to develop for each of the ten goals* an 
expected output tor each student « thus giving us a distribution of expectancy 
in each ol the several categorisations we have defined* In other words* for a 
high e'fort* »»edlus) resources* tt)edlu» school characteristic school* we night 
expect twenty percent (207.) of the students to score high for a given goal* 
thirty percent (30Z) moderately lilgh* thirty percent (307*) about the middle* 
and so on* (These are hypothetical llguros*) With this kind of an arrangctnent 
wo are no longer placed In the position of comparing a wealtliy suburban 
district with now buildings and favorable teacher/student ratios to a school 
district In a .ontmunlty with old buildings* declining population* reluctant 
taxpayers* and severe recruiting problem* In order to assure that the 
expectancy table categories accurately reflect the correlating conditions and In 
order to determine the relevance of previously Identified correlates to the 
school districts of Pennsylvania* wo are administering a measurement package 
to collect correlational data In April of 1968* It Is this package which you 
win bo reviewing tomorrow* 

Following the refinement o£ our descriptive categories through this first 
study* data on the status of accompl Istiment for each of the ten goals will be 
collected to provide norms for varying kinds of learning situations throughout 
tl\Q states Establishing norms In this manner will help us to avoid comparing 
unlike student ppulatlons* It will also provide a comprehensive picture across 
the Commonwealth of student response to the Items Judged consistent with the 
goalSf and will show what conditions are associated with varying responses* 

ERLC 



Tho Bureau of Quality Education Af;5cssmont will provide sovorfll kinds of 
sorvlccs to local schools as tho third part of tho plan. It will provide a 
nieasurotnont package for assessing students In each goal area* With the 
normative data^ It will assist teachers and administrators In looking at their 
students* output In terms of wh£»t might be expected from students In districts 
operating with similar controlling conditions. Also^ the Bureau staff will be 
prepared to assist the school staff In considering possible reasons to account 
for differences which may occur between the output of the students and their 
expectations^ regardless of the direction of the differences. The Bureau will 
provide a coordinating function to assist school districts In modifying the 
conditions or processes which lead to unsatisfactory outputs and In determining 
those which contribute to outputs greater than expectation. Furthermore^ the 
Bureau will assist local school districts In testing the effects of program 
alterations. It will assist In the development of measurement devices and In 
subsequent comparisons with appropriate expectations or norms. 

Procedures for Implement Ing tho Assessment Plan The process of 
implementing the Assessment Plan can be Illustrated by tracing one of the ten 
goals from tho beginning of measurement devclopmont^ through administration to 
utilization of results by teachers and administrators. For this Illustration 
I would like to share with you our struggles with Goal VII, Creativity. After 
abandoning, as Inadequate, two or three promising models, we have secured a 
tentative beachhead on this nebulous Island. Vo think that there arc ways to 
differentiate the degree of creativity In the expression of pumpklnness In 
children's drawlr^s of Jack o* Lanterns, and that there Is something to bo 
learned from taking a picture through a goldfish bowU In fact, on our latest 
model, the Metropolitan Museum's classic Greek Uorso, which was recently updated 
to 1917, survives as an example of somebody's creative output. 



Talking about creativity, howovor, always seoms to generate a lot of heat for 
some reason. Perhaps a slightly less controversial subject ♦ Goal II, might bo 
a less distracting illustration. 

Sow Goal II states that quality education should help every child acquire 
understanding and appreciation of persons belonging to social, cultural, and 
ethnic groups different from his own. Let us see how Goal 11 fared in the 
first phase of our plan. How does a child behave if he is fulfilling the 
requirement of this Goal? 

In cooperation with the Bureau of Research, wc began to work on this problem 
by considering existing measurement devices related to attitudes toward people 
and by expressing behavioral objectives through the construction of single items 
which were specific to some kind of behavior a student might exhibit. We 
considered and utilized items such as "How would you feel about having as a 
best friend a person whose family is much poorer than yours?" We considered 
and rejected items such as "I would ride with anybody who attends this school." 
The limitations of this single response type of question, however, led us to 
consider another type which would not only allow a student to express his 
preference for a kind of action, but also would permit him to state some reason 
for such an action. We, therefore, began to recall from our own school 
experience, and to imagine, social situations with possible kinds of behaviors 
responsive to these situations, and reasons for such responses. After a few 
attempts of this sort, we discovered an acceptance/rejection rationale emerging 
inductively from the behaviors specified through item writing. For example, if 
a Negro child enters an all Anglo-Saxon classroom, we postulate several 
possible behaviors* Some children might immediately try to get to know him 
better^ others might wait to see what he is like before being friendly, and 
still others may stay away from him. 
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It becamo aoparcnt that the reasons why a child might Innncd lately seek 
acquaintance could differ considerably. One reason could be that the child is 
genuinely open and friendly. Another child could feel obligated because he has 
been told that he should be nice to everyone. Still another might desperately 
be seeking interpersonal relationships. 

The rationale which developed, accounts for nine classifications of 
motlvest ranging from the quality of "open acceptance" as being most desirable 
to "need gratifying avoidance" as being least desirable. Falling in the middle 
range are statements such as "there are good and bad people In every group" and 
"one should be nice to everyone", exemplifying "stereotyped caution" and 
"stereotyped acceptance" respectively. 

After testing this rationale by the application of different sets of 
behaviors and through presentation and discussion with small available samples 
of students, the model evolved to the form In which it is presented In this paper. 





OPEN 


STEREOTYPED 


NEED GRATIFYING 


Acceptance 


I 1 Ike people 


One should be 
nice to everyone 


I need friends 


Caution 


I like people 
only If they 
are not too 
different 


There are good 
and bad people 
In every group 


I'm afraid to 
meet new people 


Avoidance 


I like people 
only like us 


Foreigners are 
too different 


He might be a 
bad Influence 
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Following this rationale a sot of items was then constructed and Included 
for tryout In the measurement package for Goal II. 

The second phase of our work on Goal II is yet to be accomplished. 
Ve are ready to study the influence of school, community and student 
characteristics on the incidence of the behaviors represented by measurements 
for Goal II, The output data for Goal II will be correlated with data for 
each of the characteristic variables. Those factors which seem to show the 
highest relationships will ;;hen be used in defining the categories for 
subsequent measurement on this Goal. Concurrently with this study, we plan 
to involve school personnel and laymen in further discussion of the Goal and 
its measurement implications. Norms for students within each category, so 
defined, will then be collected. When this step is completed, we will be 
ready to begin the actual task of assessing the accomplishment of progress 
toward Goal II on the part of individual school districts. The appropriate 
measurement package will be administered and classifying data collected. The 
results will be reported to each district in the form of degree of agreement 
with expectations. The expectations will be based on data from districts 
having similar conditions of operations. 

Staff from the Bureau of Quality Education Assessment will assist the 
local staff in considering the implications of the comparisons. Where 
differences too large to be attributed to chance exist, the Bureau's staff will 
work with local staff in developing hypotheses to account for the differences 
and, in the instance of positive differences, in suggesting ways to disseminate 
and further test the processes which seem most likely to have contributed to 
the successful performance. 
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In the case of negative differences a similar procedure will be followed. 
First, a hypothesis will be developed to account for the differences and then, 
depending upon whether or not the school district is in a position to directly 
influence the presumed causes, either compensatory programs or modifications 
will be suggested. Other Bureaus in the Department of Public Instruction, 
particularly the Bureau of Curriculum Development, and the Bureau of General 
and Academic Education will contribute these suggestions. Universities and 
state college instructional centers will also be involved. 

After modifications of process or program have been attempted, the Bureau 
will assist as needed in program evaluation. For example, suppose a school 
finds its students lacking openness in deai'ng with people of different ethnic 
backgrounds. Given this situation, a school might engage in some type of a 
cultural interchange through interschool visits or exchange student programs. 
The effect of this compensatory program can be evaluated against the normative 
data as well as against ^he previous measurement in the same school. It is 
expected that as more anu more schools experience this kind of assessment, 
processes which are particularly constructive will emerge and may be disseminated 
across the state to all districts who recognize their need for modifications. 

In summary, the Pennsylvania Plan for assessing educational quality consists 
of three tasks* The first of these, the selection and development of measurement 
devices, has occupied most of our time to date and is near completion for an 
April tryout* The second task of classifying data according to modifying 
conditions is also scheduled for the coming Spring. The establishment of norms 
according to these conditions will follow probably in the Spring of 1969. The 
third task of providing services to local schools will not be accomplished until 
actual assessment gets underway in the 1969-70 academic year. 



If the Plan as conceived Is realized in Pennsyl vania^ wc shall then begin to 
have objective evidence about the adequucy and efficiency of the educational 
programs offered by the public schools of the Commonwealth, with an emphasis 
on the individual school and its capacity for meeting the specific needs of 
the specific children it is to serve. There are many problems yet to be 
solved* Some of them are these: 

!• We are relying heavily on self -report for measurement in the 
less tangible areas, for example; We need to develop ways of 
assessing student output in these areas which can be considered 
both valid and economical. Perhaps self -report will meet these 
specifications, but at the very least we need to conduct 
additional validation studies. 

2« Some of the commercially published materials are at best a 

compromise fit to our measurement rationale. We need to locate 
or devise new materials which will ^better suit Pennsylvania's 
Goals. 

3. Since continued measurement is required, we must develop or 
select comparable alternate forms of all measurement devices. 

4. Many areas of the curriculum are as yet unmeasured. Example: 
Technical and vocational skills and high school electives. 

We are anticipating your understanding and assistance with these and 
other problems. We expect that we can solve them together. 

Paul B. Campbell 
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WHAT WILL QUALITY 
EDUCATION ASSESSMENT ^EASURE? 



During the last two weeks of April and the first few days of May, 
approximately fifteen hundred (1500) fifth graders and fifteen hundred 
(1500) eleventh graders In one hundred (100) schools throughout the 
Commonwealth will be administered an Assessment Package which I shall 
be discussing today and you will be reviewing tomorrow. However, as 
Dr* Campbell stressed In his presentation, adequate assessment of 
educational quality must take Into account the context in which education 
takes place. In order for us to be able to test for any relationships 
among what a student brings with him, what a school can offer him, and 
what a student achieves, we must collect what we refer to as input 
variables. These input variables are school, community and student 
characteristics which will be used as correlates of achievement in each 
of the ten goal areas. 

In order to choose from among the hundreds of available school, 
community and student characteristics, we reviewed five of the most recent 
major educational research studies. 

The first of these^ The Equality of Educational Opportunity study 
by James Coleman and others for the U.S. Office of Education, is currently 
one of the most controversial, widely discussed, and widely disputed* 
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The survey concludes that schools bring little influence to bear on a child*s 
achievement that is independent ot his background and general social context; 
and that this very lack of an independent effect means that the inequalities 
imposed on children by their home, neighborhood, and peer environment are 
carried along to become the inequalities with which they confront adult life. 
It is a grim conclusion and we shall be joining the ranks of the many who 
will try to show that schools do make a difference. 

The Fels Institute Study on urban school districts in Pennsylvania sought 
to measure differences in educational requirements of public schools in urban 
districts, compared with non-urban districts. Their findings show that 
substantial numbers of urban pupils present significantly different educational 
needs related to their socioeconomic backgrounds and such needs require new, 
expanded or intensified educational programs and services which are costly. 
Correlations between achievement and socioeconomic characteristics were most 
significant in urban districts and less significant in non-urban districts. 

The extensive testing and research studies that have been taking place 
in the state of Iowa have indicated that the educational background of parents, 
the occupational background of parents and the IQ of students correlate most 
significantly with student achievement. 

Findings from the New York Quality Measurement Project indicate that the 
variable most highly related to student achievement is the educational level 
of the mother. The New York Quality Measurement Project, begun in 1958, had 
many of the same aims and purposes as the Pennsylvania Plan. We visited 
Albany in November, hoping to be able to benefit from their experiences, only 
to find that the Project is in its last year. 
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Project TALENT is impressive not only lor its size (the sample consists 
of five percent (5%) of the population of high schools in the Country) but 
also tor its durability. The first data were collected in the Spring of I960 
and, despite the f.-'ct trhat the Project has recently moved from the University 
of Pittsburgh, they are alive and well in Palo Alto. Project TALENT contains 
a wealth of information if only one had the time to review its numerous 
volumes. We did review one of the major volumes to find that factors such as 
housing quality, teacher salaries, teacher experience and background, number 
of books in the library, and school size show relationships to student 
ach ievement . 

We are collecting other student, school, and community variables which 
do not have support in the literature as bei > significantly related to 
student achievement, mainly because they are the kinds of characteristics 
most researchers do not collect. It is our opinion that if we are to show 
that schools d£ make a difference in what pupils know, think, and believe, 
wc should pay more attention to the educational process and less attention to 
factors such as size, per pupil expenditure, and salaries, important as they 
have been shown to be. For example, we are asking teachers "What characteristic 
do you think actually counts in gaining professional recognition in this school 
system?" and 'Vhat characteristic do you think should count the most In 
gaining professional recognition in this school system?". 

We are asking principals to check a listing of innovative practices as 
they pertain to their schools, practices such as independent study, teacher 
aides, work experience programs, and instructional television. 
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Wc arc a^kinc; students ^'Whnt Is the best single way for a boy or girl to 
bo Important and looked up to by other students in this school?" 

K Dclnj; bright, well - Informed and Interesting 
2. Doing well In tests and grades 

Being an athlete or cheurloador 
U. Coming from the right family 
5. Being attractive 

At this stage we are only dealing with hypotheses* We do not know which 
school, community, and student variables will show relationships to the 
achievement of students In Pennsylvania, but we are In the process of 
collecting a great deal of information In order to find out. 

Given the Ten Goals of Quality Education, roost of which pertain to 
attitudes and opinions rather than factual knowledge, how do we discover 
whether or not students In the Commonwealth are fulfilling the requirements 
of these Goals? 

When we began six short months ago, one of the first things we did was 
contact each of the major tost publishing companies. Inviting them to send 
a representative to visit, in order for us to discuss our Project with them. 
We hoped that they would have measuring instruments, either In published form 
or In the developmental stage, that we would be able to use. Most of the 
companies accepted our invitation* However, as we began meeting with these 
test representatives. It soon became apparent that the Goals we were 
i^Lcemptlng to measure were not the subject areas into which the test publishing 
companies were putting their talents or their money. In the areas of self 
understanding, understanding others, interest in learning, good citizenship, 
creativity, vocational development, and preparation for a world of change, the 

Q test publishing companies had very little to show us. 
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Even more dis;appointlng was the fact tlint thoy did not expect to have anything 
to show us in the near future. Their efforts are going into the areas of 
scholastic achlcveiTont with the dainty as we hear with most other products on 
the market, that, "this is what the public wants*" Ihcy wished us luck and 
admired our courage* 

If we wore to measure the Ten Goals of Quality Education, wc were going 
to have to find measuring devices somewhere else, or develop them ourselves* 
Wc d£ have an Assessment Package ready for April testing* With help from the 
Bureau of Research Administration and Coordination, most of the items we 
developed ourselves* For two of the Goals we are using unpublished inventories 
being validated in university settings* 

Goal I states that Quality Education should help every child acquire the 
greatest possible understanding of himself and an appreciation of his 
worthiness as a mcmbcx of society* We expect to find differences in the w.n^ 
a student feels about himself, about his school and social situations* The 
literature suggests that how a student sees himself, his adequacies, his 
inadequacies, his values, has a strong relationship to how he achieves in 
school* The student with the positive self concept will be able to focus 
attention on classroom activities* He will bo able to accept and recognize 
his weaknesses, he will not feel less worthy as a person because of thom and, 
most important, ho will be able to capitalize on his strengths* 

A second dimension to bo moasuroci in Goal I is based on tho theory that 
the sense of control which a student Cools concerning his environment is 
significantly related to how he performs in school* Does the student view his 
world as one In which he will be able to fulfill his hopes, his ambitions? 
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Can ho feci personal responsibility for planning his future £ind proceed vtth 
direction and purpose^ discounting for the taost pdrt» the cletoent of luck as 
the deciding factor? Can he recognize that everyone faces barriers .ind 
1 Itnitations and these need not be immovable obstacles? 

Goal 11 states that Quality Education should help every child acquire 
understanding and appreciation of persons belonging to social, cultural, and 
ethnic groups different from his ovn# Correlating the results of our Coal It 
questions with the student, school and community Input variables will allow 
us to test certain assumptions* For example, where a student body is 
heterogeneous with regard to educational and occupational background of parents, 
are these students more likely or loss likely to show greater understanding 
and appreciation of persons belonging to social groups different from theirs? 

Docs the growth, stability or decline of a community show any relationship 
to a student*s understanding and appreciation of others? 

Questions for Goal II center around social class, religious and ethnic 
differences* Our questions are worded In such a way as to not mako references 
to any particular race, religion or social class. Although our sample schools 
have been, for the most part, positive and receptive to these kinds of questions, 
we are aware that It Is In this area where apprehension and doubt arc most likely 
to exist, and we look for your comments and advice In those matters* 

Goal III states that Quality Education should help every child acquire to 
tho fullest extent possible for him mastery of the basic skills In the use of 
words and numbers* There is no difficulty In finding a reliable and valid 
Scholastic Achievement Test* Tho difficulty arises when one tries to choose 
from among the many* 
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Tho problofii is further cotapoundod when ono discovers th.it the typic^il Scholadtic 
Achicvctti(?nt Tedt takod trom three to five hours to ^idminidtcr* When stis 
considered the fcict that our aim to administer the full battery in no cnore 
than Cive hourd^ it b<^cas}c obvious that ve could not dimply choose and administer 
a typical Scholastic Achievement Tost. 

A survey conducted by the Bureau of Guidance Services reveals that 
practically all students in tho Commonwealth are administered an achievement 
battery in the elementary grades as well .is the secondary level* Upon reviewing 
tho survey more carefully, wo discovered that the number d£ achievement batteries 
that wore used centers around five or six major onos. Wo then diverted our 
attention to tho possibility of using existing achievement tost scores. Tho 
problem then became one of 'MIow do wo compare* say» scores from the Stanford 
Achievement Test» with scores from the Metropolitan Achlovoroont Tost» or with 
scores from tho School and College Ability Tests?" TIio opinions of the test 
publishers wore not very cncouragingt but wo have not abandoned the idea* 

For our April testing^ howovor^ tiarcourt Braco and World is building us a 
survey tost based on the Stanford Achlc^^cmont Battery which will yield a total 
score and take just one hour to administer. 

Goal IV states that Quality Education should holp every child acquire o 
positive attitude toward school and toward tho learning process* In tho 
positive aspect of this dimension tho student will vlow education as bolng 
holpful toward obtaining a job» making decisions as a voter » enjoying leisure 
time activities^ kooplng Informed of world events^ participating offoctly In 
community affairs^ and maintaining a homo of his own. Ho will express tho 
Interest and desire to graduate from high school. He will consider school and 
learning lntorostlng» valuable^ plcasMt» s\d active. 
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Ho will express the opinion that learning does not end whore formal education 
ends, and will express the desire to return to some kind of educational 
setting from time to time In his adult life* 

Correlating the results of the Goal IV question*- with the Input variables 
vlll allow us, for example, to test whether or not students show tnore positive 
attitudes toward learning where teacher attitudes toward learning, Innovation, 
and education are more positive* Does the slse of a class or a school have 
any relationship to student attitudes toward learning? Where college attendance 
rate Is higher, arc student attitudes toward learning more positive? 

Coal V states that Quality Education should help every child acquire the 
habits and a' tltudes associated with responsible citizenship* It seems as 
difficult to find an Inventory to measure responsible citizenship as It Is to 
find responsible citizens these days* We not only have an Inventory, but a 
responsible citizen as well, for In this short time, Dr* Ernest Peters from 
the Bureau of Research has not only developed the Pennsylvania Citizenship 
Assessment but has also been able to administer It In two school districts 
to collect reliability data* Students will respond to questions such as: 

When my parents give me a Job to do, I do It* 

If a store clerk gave me too much change, I 
would keep the extra money* 

A man has to cheat a little If he wants to get 
ahead In this world* 

For Goal VI, understanding good health habits and the conditions necessary 
for the maintaining of physical and emotional well-being, we reviewed the 
relatively few standardized Instruments and selected the California Health 
Behavior Inventory* 

Er|c 26 



In order to expand our mGasurctnont in this area, wc are considering the use of 
a teacher check list, whereby the teachers can check personal observations of 
grooming and health habits. We, perhaps, can also utilizo health records which 
are available in every school. 

Creativity, Goal VII, was defined by the Quality Education Committee as 
including worthwhile activities that the student initiates and pursues on his 
own, activities producing an outcome which is perceived by the child himself 
and by others as a contribution to some part of this world. Our concept of 
measurement in this area rests first on the theoretical basis that there is 
a student potential for creative output. Second, that there are conditions 
under which creative output is more likely to occur and, finally, that there 
are characteristics which are common to creative production. This theoretical 
construct thus includes the idea of the student's potential, the conditions 
under which he works and the quality of his output. 

For the elementary student, Thomas Rookey, a research intern, had developed 
the Pennsylvania Appraisal of Creative Tendencies. Students will respond to a 
five point scale ranging from strongly agree to strongly disagree to questions 
such as: 

It's great fun to make up stories. 

It would be a waste of time to take a 
photograph through a fishbowl. 

You should not change the rules of the game. 

The secondary student will check "yes" or "no" to a list of seventy-nine 
(79) activities, each of which has been assigned a quality rating. He will 
also be given a series of questions as a measure of creative potential. 
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Our efforts in this area, as well as everyone else's efforts in this area, 
are strictly experimental. There are many who maintain that creativity cannot 
be measured. In fact, there are those who maintain that many of our goals 
cannot be measured. At this time we are taking a more optimistic view and are 
operating on the opinion that using less than perfect instruments is better than 
limiting our evaluation to t'lose areas where more perfect measuring devices do 
exist. The Committee on Quality Education was very emphatic in their defining 
of Quality Education to include the many attitudes, habits and accomplishments 
which are not easy to measure. To limit our measurement to those one or two 
goals which are relatively easy to measure would not be, in our opinion, 
fulfilling the wishes of the Conunittee nor the purposes of the Act. 

Goal VIII states that Quality Education should help every child understand 

the opportunities open to him to prepare himself for a productive life and should 

enable him to take full advantage of these opportunities'. Guidance counselors 

have long been interested in the area of vocational development and there have 

been many theories, many inventories, and many pages written concerning this 

process. However, the majority of this work has been with secondary students. 

Research in the area of vocational maturity with elementary students has been 

» 

practically non-existent. We considered the question, "How can fifth graders 
reasonably be expected to behave in matters concerning choosing a life's work, 
choosing a life's style, making decisions and preparing for independence?" I 
had the pleasure of raising some of these questions with small groups of fifth 
graders in the surrounding area, and based on our experience with these students^ 
together with the position taken by the Bureau of Guidance Services, we are 
hypothesizing that fifth graders are aware of different kinds of work and workers 
and have a growing understanding of the relatedness of educational and vocational 
opportunities. 
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By the eleventh grade the more mature student will show involvement in 
the vocational choice process by actively seeking information about the world 
of work. He will take the responsibility for making a career decision. He 
will not depend upon others to make these choices for him. Finally, the 
student will base his career choices upon his interest, ability, and aptitude 
using a realistic appraisal of his potential as a basis for his decision 
making. 

Goal IX states that Quality Education should help every child to 
understand and appreciate as much as he can of human achievement in the 
natural sciences, the social sciences, the humanities, and the arts. Our 
items for Goal IX attempt to measure the understanding of human achievement, 
but we have neither found nor developed a measure of appreciation of human 
achievement. 

By the time we reached Goal X, I sometimes think we ran out of steam, or 
ideas, for the majority of items we are using to measure in this area are being 
used to measure in other areas as well. Part of our difficulty may also be 
due to the fact that our understanding of Goal X seems to overlap with Goals 
IV, VII, and VIII, However, Dr. Martin Yanis from the Bureau of Research has 
developed several items which are unique. Goal X states that Quality Education 
should help every child to prepare for a world of rapid change and unforeseeable 
demands in which continuing education throughout his adult life should be a 
norma 1 expe c t a t i on . 

Although being able to assemble this Assessment Package for your review 
tends to give us a feeling of having something completed, we must keep in mind 
that this Assessment Package is just a beginning. Test development for the 
Quality Education Project must be oi\e of ongoing revision, and constant refinement. 
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Joan S. Beers 
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PROGRESS REPORT ON THE 
NATIONAL ASSESSMENT PROJECT 



Educational evaluation has been undergoing profound changes in both 
theory and practice in the past two decades. These fundamental develop- 
ments are to be found in the several major uses of educational measurement 
and evaluation, such as in guidance, admission and placement, and in the 
awarding of scholarships; in the diagnosis of student learning; In 
developing programs, courses. Instructional procedures, materials, and 
equipment; in monitoring and guiding educational programs in the school and 
in assessing the progress of education to provide bases of public under- 
standing and public policy. These changes have arisen both from the changing 
educational situation and from the development of new knowledge and technology 
in education. 

The applications of science and technology in agriculture, industry, 
defense, coinmerce and the health servip.es have shifted the nature of human 
occupations from those largely based on physical strength and manual 
dexterity to those involving large components of intellectual activity and 
social sensitivity and skills. When I was born, thirty-eight percent (38%) 
of the U.S. Labor Force was engaged in agriculture. Now only seven percent 
(7%) is so employed. Then, another twenty- three percent (23%) were utilized 
as unskilled labor in non-farm occupations. Now only five oercent (5%) is so 
employed. Very few young people can find jobs who have not attained the 
functional literacy represented by the average achievement at the end of the 
fifth grade. 
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At the other extreme, the great employment opportunities are in science, 
engineering, education, the health services, recreation services, social 
services, management, and accountings In our time, the role of the school 
has shifted from that of selecting out a small percent of the pupils for 
more advanced education, while the others dropped out and went to work, to 
that of reaching every child effectively to enable him to go on learning far 
beyond the expected level of twenty-five (25) years ago. The task of the 
college is not to find a favored few, but to identify a wide range of potential 
talents and help each student to realize this potential both for his own self- 
realization and to meet the ever-increasing demands of a complex technological 
society. This changing situation requires new instruments and new procedures 
of educational evaluation and measurement. 

New knowledge and technology in education are also influencing evaluation. 
For example, the recent findings of many studies regarding the powerful effects 
of the student's home, culture, and community environment upon his learning 
have clarified the need for evaluating these factors in order to guide and 
improve education. As another example, a series of investigations, like those 
of Newcomb and Coleman, have shown the strong influence of peer group attitudes, 
practices and interests upon the learning of its members. This indicates the 
need for evaluating the nature, direction and amount of peer group influences 
In developing effective school programs. 

These highly significant changes have brought about new conceptions and 
new practices in several areas of education evaluation. I shall report on 
the developments sometimes called the "National Assessment.** 
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The purpose of the project on assessing the progress of education is to 
provide that intelligent lay public with census-like data on the educational 
levels of important sectors of our population in order to furnish a dependable 
background of information about our educational attainments, the progress we 
are making and the problems that we still face in achieving our educational 
aspirations. Since education is now being recognized as a necessity for every- 
one in order to participate in our complex social, civic and industrial life, 
the public is asked to support programs for extending both the quantity and 
quality of education. Without perspective regarding the progress we have made 
and the difficult tasks we face, our citizens have an inadequate basis for 
making judgments. As a result, decisions are frequently made on hearsay or 
widely published assertions rather than on a reasonably clear picture of the 
educational situation. 

Recognizing the lack of comprehensive and dependable data about the 
educational attainments of the various sectors of our population, Carnegie 
Corporation, a private foundation, in the Summer of 196A appointed an 
Exploratory Committee on assessing the Progress of Education, I was asked to 
serve as Chairman. The Committee's Staff Director, for an initial six (6) 
month's period was Stephen Withey of the University of Michigan. Then we were 
able to obtain Jack Merwin on leave for two years from the University of 
^ Minnesota and since the Summer of 1967, Frank Womer of the University of 
Michigan has been Staff Director for the project. 

The Committee's assignment is to confer with teachers, administrators, 
school board members, and others concerned with education to get advice on the 
way in which such a project may be constructively helpful to the schools and 
avoid possible injuries. 
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The Committee is also charged with the development and tryout of instruments 
and procedures for assessing the progress of education. The Committee has 
been working on these assignments for more than three years. In 1966, the 
Fund for the Advancement of Education joined in supporting the project. 

Since several reports have been made on the purpose and plan of this 
assessment project, I shall only outline them briefly. The conferences with 
teachers, supervisors and administrators resulted in strong recommendations 
that the initial assessment should include more than the 3R's and that 
ultimately all important educational areas should be included. Hence the 
four test construction agencies, American Institutes of Research, Educational 
Testing Service, Psychological Corporation and Science Research Associates 
have contracts to develop assessment exercises in reading and the language, 
arts, mathematics, science, social studies, literature, music, the visual 
arts, citizenship and vocational education. New instruments had to be 
developed because currently available tests are constructed to provide 
dependable means or median scores for groups but do not furnish information 
about the achievements of the lower and upper ends of school populations. Yet 
an assessment of educational progress should provide information about the 
progress being made by the less advanced and the more advanced children as well 
as those who are in the middle of the distribution. 

Furthermore available tests are constructed to be given to every pupil 
normally in a classroom setting. To assess the progress of various segments 
of our population it is not necessary for every child to take every exercise. 
Samples of the population can furnish a dependable picture of the progress on 
the vhole. 



Some exercises can be performance tasks, some can involve observation or 

individual interviews as well as paper and pencil tests. This permits a 

more adequate assessment of the progress of our children, youth and adults 
towarr* the several major objectives of education. 

The assessment will report on the educational achievement of two 
hundred and fifty-six (256) parts of the total U.S. population. These 
two hundred and fifty-six (256) parts are obtained by sampling separately 
each of four age groups: 9 year olds, 13 year olds, 17 year olds, and 
adults between 26 and 35 years of age; each of four regions: northeast, 
southeast, midwest and farwest; each of four sizes of communities: big 
cities, small cities, suburban areas, and rural -small town areas; two 
socioeconomic levels and the two sexes. 

To summarize the educational attainments of these several populations 
it is not necessary to compute test scores. Instead, the following sorts 
of things will be reported: 

For the sample of seventeen-year-old boys of higher socio- 
economic status from rural and small town areas of the Midwest 
region, it was found that: 

93% could read a typical newspaper paragraph like the 
following. 

76% could write an acceptable letter ordering several 
items from a store like the following. 

52% took a responsible part In working with other 
youth In playground and community activities like the 
following. 

24% had occupational skills required for initial employment. 
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It Is antlcioated that the assessment would be In charge of a commission 
of highly respected citizens* They and the commission staff would prepare 
reports of the findings of the assessment, much as we now obtain reports of 
the findings every ten years of the decennial census. These reports would 
be available to all people Interested In education, providing them In this 
way with significant and helpful Information on what has been learned by 
each of the two hundred and flfty-slx (236) populations* In subsequent years, 
the progress made by each of these populations since the preceding assessment 
would also be reported* 

In conducting this project, a number of technical problems have been 
encountered that are relatively new to educational measurement* In dealing 
with them, the Committee and its staff have not only had the benefit of the 
experience, intelligence and skills of the contractors but we have had very 
competent technical advice* The technical advisory committee consists of 
John W, Tukey, Chairman, who heads the Department of Statistics at Princeton 
University, Robert Abelson, Professor of Psychology at Yale, Lee Cronbach, 
Professor of Educational Psychology at Stanford and Lyle Jones, Professor of 
Psychology at the University of North Carolina* 

Among these technical problems are the following: How do we communicate 
with children who come from home backgrounds of limited education and who are 
unable to read? What kinds of errors are Introduced In administering the 
same exercises to some children Individually and to others In groups? What 
biases, If any, are Introduced by administering assessment exorcises out of 
school In comparison to In-school administrations? How can a probability 
sample of children, youth and adults be chosen that represents the various 
sectors of the population and at the same time, minimizes costs of administering 
O the assesstr^nt exercises? 
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These arc a few illustrations of the tiany technical questions that have arisen* 

The National Opinion Research Center has conducted a study Cor us, testing 
the feasibility of administering the exercises to adults in the home and also 
in small group settings* Surprisingly enough, adults are interested in taking 
the exercises, and sessions of one and a half (ih) hours with each adult are 
quite feasible. 

The Research Triangle Institute of Duke University and the University of 
North Carolina are not only designin^^ and testing the sampling procedure for 
identifying those who are to be assessed but the Institute also is conducting 
comparative investigations of assessing youth in school and out of school* 
The American Institutes of Research are conducting studies involving individual 
administration of exercises followed by interviews with children tested to 
check the comprehensibil ity of directions and wording of exercises* They will 
also obtain findings regarding pupil motivation in taking various kinds of 
exercises* 

Educational Testing Service is conducting extensive tryouts to check the 
feasibility of various types of exorcises and a* so to obtain indications of 
the range of achievement levels for which given exercises are appropriate* 
This special study is a large and comprehensive one* 

These illustrations should suggest the kinds of published reports that 
will bo available in connection with the study of those technical questions* 
They will be of value to students of educational measurement concerned with 
a wide variety of uses of evaluation* 

In addition to furnishing greatly needed information to the general public, 
thoro arc some special values of this project which \cre not anticipated when 

,^^.it was begun* 



One of these Is tho dovolopmont oi tisscssmcnt cxcrcl&es approprl«3tc 
for those children and youth In ooch age group who «iro among the lovost 
third In achlovenK-nt. Secauso currently used achievement teats arc focused 
on average pcrf o)*niance » their exercises are concentrated at a ciiddle band 
of difficulty. Only about five percent (57J of the exorcises in cotjoianly- 
uscd tests are relevant to what Is learned by students who are In the lowest 
third of their group* As a result, wo have no way of evaluating the progress 
being made by this lowest third, A somewhat similar situation exists In the 
case of the highest third. 

It was difficult for our contractors to construct valid exercises for 
both tails of the distribution because it has never been done before* The 
first efforts proved on tryout to be unsuccessful, but by obtaining the 
services of teachers and supervisors who are working effectively with children 
from low socioeconomic backgrounds to help In developing exorcises and by 
careful revisions In the light of tryout results, It has been possible to 
produce assessment exercises that can appraise the educational progress of 
children In all three segments of the distribution* This will bo of great 
value In the future as we seek to evaluate the of foctlvoness of programs 
that have been dcvolopfd to serve special school populations* 

A second kind of value of the uitsossment lies In tho demonstration It 
will furnish of a wider range of useful evaluation procedures and studios 
than those that have commonly buen employed* School testing programs now 
are concentrated on obtaining moan or median values for classes and schools 
and with tho reporting of relative standings of Individual pupils In standard 
scores, quartlles, percentiles, and the like* These have very limited usefulness 
In evalyatlng programs, curriculum materials, teaching procedures, learning 
and Instructional devices and so on* 
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As a proteshion uc ^Ivo lip scrvlco to appraising progress tow.ird all 
laiportant educational objectives using a variety of evaluation techniques, 
but in lawtf wc do not comonly cover all our iixiport«int aims, nor do vo 
usually depend on techniques other than paper and pencil tests* The assessment 
project reminds us that we can make appraisals In terms ol our several objectives, 
and It Is demonstrating ore way of doing It* It Is showing us that we do not 
need to have every pupil take short, undefined or undesignated tests* It Is 
demonstrating the feasibility of using samples of students so that the range 
and depth of evaluation can be greatly extended* It Is showing us the possibility 
of systematically sampling the student behavior we are seeking to develop, at 
the particular stage of each child's progress, through the use of a range of 
devices Including not only paper and pencil exercises but also observations. 
Interviews, questionnaires, performance tests, and samples of pupil products* 
In my opinion, this contribution to the development of more adequate evaluation 
within our own schools may be the most Important contribution of the current 
assessment proJcct« 

In summary, assessing the progress of American education provides a means 
of helping the public understand the Instructional purposes, achievements and 
progress of our schools and furnishes the professional staff with means for 
evaluating much more comprehensively than In the past, the effectiveness of 
educational programs designed to serve particular purposes* Both cho results 
and the procedures of the assessment project can make a constructive contribution 
to education* 

Ralph W. Tyler 
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Sr^ri>RY OF CONSENTS AND Sl'CGESTIONS 



Made by thf^ Advisory CommittGc for the 
Piuroou of Quality Education Assessment 
at its First Mootins^ on February 1 and 2, 1968 



Followin;; the presentation by members o£ the Bureau on 
Thurf;dny nfletnoon, the Committee divided into two groups to 
react to the Pennsylvania Assessment Pl.in* 

Concern centered about the sampling procedure for the 
April testing. It was explained that the sample of one hundred (100) 
schools was randomly selected from within a total list of school 
districts stratified by market value of property and size of school 
population. One of the purposes for the April testing is to 
identify other significant dimensions by which school districts can be 
characterized. The data from the April testing will form the bases by 
which schools will be classified for the normative study, scheduled 
for the Spring of 1969, 

For the most part, the Committee was favorable and supportive 
of the efforts of the Bureau, while at the same time cognizant of the 
Inherent difficulties underlying any large scale assessment plan. 
Committee members lauded the work of the original Committee on Quality 
Education for emphasizing that factual knowledge is only one aspect of 
student achievement and that the feelings, attitudes and opinions a 
student learns are deemed equally important. Most of the members 
voiced their approval of the efforts of the Bureau to measure 
achievement in the attltudinal goal areas. 
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It was brought out» however, that one of the most significant draw- 
backs of the Plan is the reliance on paper and pencil, self report 
Instruments. Several members of the Committee expressed the fact 
that the process of assessment can take other forms, such as 
observational techniques. Some Committee members expressed the 
opinion that measurement in Goal II, for example, might be more 
valid if the other techniques were used. 

The Friday morning sessions were devoted to a review of the 
assessment package. With the questions in front of them, the 
Committee members were able to be more specific in their reactions 
to the measuring instruments. Their major concern centered about 
the language of the items-language in terms of difficulty level 
and in terms of semantics. 

Although the Bureau applied the Flesch Reading Ease Chart 

to the majority of items for fifth graders, some of the Committee 

f 

members felt that some fifth graders might not understand the 
terminology of certain items. Reading ease is a particularly 
important consideration. As Dr. Tyler noted, where a student does 
not understand the language of the questions, we cannot reliably 
assess the dimension. 

The concern with semantics involves differences in meanings 
among different social classes. For example, the word "scolded" may 
have a different connotation for the middle class youngster than it 
has for the underprivileged youngster. 

ERLC 



Some Comnii;ti;ee members suggested that our items be reviewed by, for 
example, students, and, perhaps, teachers of the underprivileged, to 
detect such nuances in the terminology. 

One further concern related to the wording of certain items in 
a manner which seemed to imply a value judgement of the behavior in 
question* 

Other Committee members were concerned with "establishing a 
climate" for administering the assessment package- The consensus of 
opinion was that the Bureau should establish a list of "guidelines" 
for teachers concerning the nature of the assessment and its purposes. 
They maintained that although the actual testing will be done by 
monitors and not by teachers, it is the teachers, the day or so before 
the test date, who will influence the attitudes with which the students 
will approach the testing situation. 

The more specific item revision suggestions involving grammar, 
punctuation and syntax, although not specified in this report, are 
also being utilized in the refinement of the instruments. 
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CONCLUSION 



With the administration of the assessment package in the Spring 
of 1968, the first phase of the Pennsylvania Plan for the assessment 
of educational quality will be realized. The Bureau will then begin 
the task of analyzing the data in order to determine which school, 
community, and student characteristics are significantly related to 
the achievement of Pennsylvania students in terms of the ten goals 
of quality education. 

At another conference tentatively scheduled for the Fall of 1968, 
we shall be examining in detail the results of the April testing* In 
light of our findings, we shall be considering the proposed goals of 
education in the Commonwealth and again shall be reviewing the 
assessment package. 

We are confident that the Committee, together with educators and 
other citizens of the Commonwealth, can make a significant contribution 
to the vital task of assessing the quality of education in the public 
schools of Pennsylvania, 

Neal V. Musmanno 
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"To be important, an educational outcome 
must make a difference. If it makes a 
difference, the basis for measurement 
exists." R.L. Ebel 



